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ABSTRACT: According to the Ministry of Manpower, falling and slipping accidents are one of the most common 
accidents in addition, falls from heights (FFH), including accidents during scaffolding work, are still a major 
cause of death in the construction industry. Regular safety checks are currently being carried out on construction 
sites, but scaffold-related accidents continue to occur. Sensing technology is being attempted in many industrial 
sites for safety monitoring, but there are still limitations in terms of the cost of sensors and object detection, which 
are limited to certain risks. Therefore, this paper proposes a deep learning-based pose estimation approach to 
identify the risk of falling during scaffolding work in the construction industry. Through analysis of the correlation 
between unstable behavior during scaffold work and the angle of keypoints of workers, the proposed approach 
demonstrates the ability to detect the risk of falling. The proposed approach can prevent falling accidents not only 
by detecting construction site workers, but also by detecting specific risky behaviors. In addition, in limited work 
environments other than scaffolding work, the information on unstable behavior can be provided to safety 
managers who may not be aware of the risk, thus contributing to preventing falling accidents. 
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1. INTRODUCTION 


According to the Ministry of Manpower, falling and slipping accidents are one of the most common accidents in 
addition, falls from heights (FFH), including accidents during scaffolding work, are still a major cause of death in 
the construction industry. Working at height on construction sites is associated with a major risk of falls that must 
be properly managed to prevent injury and death. 


Statistics from the U.S. Department of Labor in 2021 indicate that nearly one in five workplace fatalities occurred 
in the construction industry, with over a third of these attributed to falls, slips, and trips. The construction industry 
was responsible for 46.2% of all deaths from falls, slips, and trips that year(U.S. BUREAU OF LABOR 
STATISTICS, 2023). 


Based on data from Singapore's construction sector, construction accidents have been on the rise from 2020 to 
2022, accounting for 171 incidents (26% of total accidents). Of these, FFH accounted for 55 incidents (32%), 
while slips and trips were responsible for 27 incidents (15.8%) (MOM, 2022). 


Meanwhile, in South Korea, despite the implementation of the Major Accident Act in January 2022 aimed at 
reducing fatalities among construction workers, there has been a 0.01% increase in such incidents as of March 
2023. Specifically, in 2022, out of 539 deaths, 268 were attributed to "falls," marking the highest cause. These 
statistics reconfirm that the construction sector is hazardous, with a pronounced risk of fall incidents. 


While various personal protective equipment and safety devices have been developed to prevent accidents, a study 
by (Xia et al., 2018) highlighted that one of the main causes of fall incidents is the absence or insufficiency of 
worker behavior supervision. Information on the posture of construction workers can serve as valuable data for 
evaluating safety and productivity (Xu et al., 2022). Several studies have been driven based on this perspective. 


(Khan et al., 2021) researched methods to detect unsafe behaviors of workers using IMU sensors, specifically by 
assessing the angle of attached hooks. Enrique (Valero et al., 2016) used wearable devices based on IMU sensors 
to analyze movement data and detect unsafe postures. There have also been significant strides in using computer 
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vision for safety research. (MassirisFernandez et al., 2020) proposed a method to assess work risks by analyzing 
joint angles of workers under limited visibility conditions. 


Deep learning methodologies have also been applied to safety research. (Khan et al., 2021) utilized deep neural 
networks to classify and segment worker movements, detecting unsafe behaviors on moving platforms. Pinsheng 
(Duan et al., 2023) introduced a technique using OpenPose to detect personalized stability based on the individual 
characteristics and habits of workers at height. 


However, a gap has been identified in safety monitoring research following the Standard Operating Procedure 
(SOP). This study aims to address this gap by proposing a method to monitor unsafe postures of workers during 
scaffolding operations based on their shoulder joint angles, in accordance with the SOP. 


Furthermore, by actively incorporating such innovative technology at construction sites, it is possible to enhance 
worker safety and improve productivity. For instance, these posture detection systems can support workers in 
maintaining safe working postures and preemptively detect unsafe conditions to prevent accidents. This not only 
elevates safety standards in the construction industry but also enhances work efficiency while reducing fatigue and 
injuries during operations. 


Moreover, the practical utilization of such technology in construction sites can lead to adherence to project 
schedules and cost savings. Real-time posture detection and analysis at the site are expected to contribute 
significantly to enhanced productivity and the creation of a safe working environment in the construction industry. 


2. LITERATURE REVIEW 


The construction industry has been a focal point of research on workplace safety, largely due to the high frequency 
of fall-related accidents (MOM, 2022; U.S. BUREAU OF LABOR STATISTICS, 2023). With a significant portion 
of accidents attributed to falls from heights (FFH), slips, and trips, it is clear that this is an area in need of targeted 
interventions and technological innovation. 


Efforts have been made to mitigate the risk of falls in the construction industry by developing personal protective 
equipment and safety devices. However, studies suggest that the root cause of fall incidents often lies in inadequate 
supervision of worker behavior (Xia et al., 2018). Construction worker's postures are a valuable source of stability 
and productivity information that can be used to manage construction sites (Xu et al., 2022). 


Monitoring changes in a worker's posture can provide information about whether the worker is in a safe or unsafe 
position. Numerous studies have been conducted to detect and analyze unsafe behaviors of workers using this 
approach. For instance, (Khan et al., 2021) conducted a study using IMU sensors to determine unsafe worker 
behavior through the angle of secured hooks. Similarly, Enrique (Valero et al., 2016) used wearable devices based 
on IMU sensors to measure movement data, detecting and characterizing the unsafe postures of workers. Other 
researchers have also leveraged computer vision to detect and analyze worker posture information. For instance, 
(MassirisFernandez et al., 2020) evaluated job risk by inferring joint angles of workers under limited field of view 
conditions based on computer vision. 


Deep learning has also found its applications in this context. (Khan et al., 2021) identified unsafe behaviors on 
mobile platforms through object correlation detection using deep neural network-based worker classification and 
segmentation. Also, (Duan et al., 2023) proposed a personalized stability detection technology based on high- 
altitude workers' body posture patterns by detecting individual physical characteristics and habits using the 
OpenPose method. 


Amidst these technological advancements in detecting unsafe behaviors, there's a noticeable gap when considering 
overlapping or closely interacting workers in construction sites. Addressing this complexity, (Park et al., 2023) 
introduced a method for detecting small and overlapping workers at construction sites, emphasizing the intricate 
interactions and spatial relations of workers in crowded environments. Demonstrated significant potential in 
ensuring workplace safety by identifying such challenging scenarios with higher accuracy. 


While these prior studies have inferred the results of unsafe worker behaviors using sensors and evaluated worker 
stability, they often lack in terms of monitoring safety during scaffolding work as per Standard Operating 
Procedure (SOP). Therefore, this study aims to monitor unsafe postures by detecting shoulder joint angles of 
workers during scaffolding work based on SOP, as an effort to prevent fall accidents in this context. 
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3. METHODS 
3.1 Research framework 


In the construction industry, the safety of workers holds paramount importance, especially with the rising concerns 
related to fall-related accidents. Traditional methodologies have primarily leaned towards sensor-based approaches 
to address these concerns. However, there's a growing interest in methodologies based on computer vision. In this 
framework, we utilize the YOLOv7 Pose algorithm to detect and assess the shoulder joint angles of construction 
workers to discern potential hazardous situations. As illustrated in Fig. 1 the framework to monitor the stability of 
construction workers working at elevated heights consists of five modules. First, data is collected from the 
construction site, encompassing a myriad of movements by the workers. From the collected video data, essential 
frames are extracted and major regions pertaining to the workers are labeled accordingly. Next, the YOLOv7 Pose 
algorithm is employed to detect the skeleton of the workers. Special emphasis is laid on the skeleton information 
around the shoulder joint to calculate the angle of the shoulder joint. Subsequently, an analysis is performed to 
determine if the current angle of the worker's shoulder joint poses any risk in comparison to a set standard angle. 
If any hazardous posture is detected, real-time alerts are displayed, urging the implementation of necessary safety 
measures. 


Data set 


Validation 


Fig. 1: Research Framework. 


3.2 Identification method 


Ensuring the safety of construction workers in high-risk environments, such as working at heights, has always 
been a top priority in the construction industry. Falls from heights are a common type of accident on construction 
sites, and many of these incidents occur when workers lose their balance and become unstable. In particular, 
scaffold work, which is frequently performed by construction workers, is associated with a high number of 
accidents. Therefore, in this study, the OpenPose method, which detects human postures from images and videos, 
was used to detect and analyze the postures and joint information of scaffold workers. The algorithm, trained using 
the MS COCO dataset and videos captured at construction sites, detects and represents human posture as shown 
in Fig. 2 It identifies a total of 17 joints, numbered from 0 to 16, as illustrated in Fig. 2, right image, and provides 
information about each joint. Table 1 illustrates unsafe postures that can occur during scaffold work. While various 
postures are possible during scaffold work, such as sitting, standing, bending, or bowing the head, there is a posture 
where both arms are raised simultaneously, lifting both the left and right shoulders, with a shoulder angle of less 
than 90 degrees. In this study, a posture involving the simultaneous elevation of both arms and shoulders with an 
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angle less than 90 degrees is designated as an unsafe posture. 


Fig. 2: Joint position information. 


Table 1: Unstable behavior criteria for scaffolding operations 


Unstable behavior description Angle name Angle numbers Range Threshold 
Arm rasing Left-Shoulder #756 0°~270° ZLS < 90° 
Right-Shoulder 28,6,5 0°~270° ZRS < 90° 


3.3 Feature extraction 


Algorithm 1 demonstrates a method for representing risk alerts based on angle calculations. This is employed to 
indicate stability alerts based on the angle when a worker raises their arm during operations. The algorithm 
consistently computes risk alerts based on shoulder angles, thereby continuously detecting workers and providing 
real-time notifications for risks associated with unsafe postures. It processes the input, detecting individuals in the 
video, and then delineates Bounding Boxes (BBox) and keypoints. Subsequently, it performs ongoing calculations 
for the angles of the R (Right) and L (Left) Elbows and R, L Shoulders. When the angle falls between 90 and 180 
degrees, a warning notification is sent, while angles below 90 degrees trigger an Unsafe posture notification, which 
is visualized and transmitted. 
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Algorithm 1: Finding Angle between Three Points 


Input: Detected Workers 


Output: angle 


— 


: Extract coordinates for p1, p2, p3 from kpts. 


N 


: Calculate angle using atan2. 


U 


: if angle >= 180.0 THEN label = '[safe!]', color = blue 
4: else if 90 < angle < 180 THEN label = '[Warning!]', color = yellow 


5: else label = '[Unsafe!]', color = red 


O 


: if draw then visualize angle, label on image. 


N 


: Return angle. 


Fig. 3 demonstrates the method for detecting unsafe postures when the algorithm is applied during video recording. 
The YOLOv7 algorithm was utilized for its speed and capability to detect individuals while simultaneously 
detecting joints. The worker's joints are divided into 17 distinct points, from the head to the legs, each assigned an 
ID. To identify unsafe postures during scaffold work, the angles of the shoulders are crucial when the arms are 
raised. Therefore, angles for the joints L-Shoulder(5) and R-Shoulder(6) were detected and calculated. For angle 
calculation, a line connecting L-Shoulder(5) and R-Shoulder(6) was used as the baseline, and vectors were applied 
to the Elbow joint for angle calculation, as the calculation required angles for external rotation at each shoulder 
joint. Following the KOSHA GUIDELINE, when a worker raises their arm, an unsafe posture is determined based 
on a 90-degree threshold. The system alerts the user with three levels: over 180 degrees for safe, between 90 and 
180 degrees for warning, and below 90 degrees for unsafe. 


angle > 180° 
[Safety 


90° < angle < 180° 


angle < 90° 


Unsaticty 


Fig. 3: Joint information and how to calculate shoulder joints 
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4. RESULTS 


(b) 


Fig. 4: Worker safety monitoring results during scaffolding operation. (a) Scaffolding installation work image 
taken diagonally, (b) Scaffolding installation site work image taken from the front 


In Fig. 4 (a) presents the results of detecting a worker engaged in scaffold installation captured from a diagonal 
angle. When the worker raises their arms, the proposed angle-based criteria triggers a [Warning!] alert, indicating 
the detection of the worker and the activation of the alarm system. In (b), an image captured from the front 
showcases the construction site, not only identifying scaffold workers but also other workers engaged in different 
processes. As these workers are not in the arm-raised posture, no warning alerts are generated. 


The proposed method for detecting unsafe postures during elevated work involves worker detection on 
construction sites and shoulder joint angle assessment for safety monitoring. This approach aligns with safety 
posture guidelines applied when workers raise their arms during elevated tasks, providing real-time warning 
notifications. This experiment validates the robustness of this study in facilitating easier site supervision and 
monitoring by administrators in complex and chaotic construction environments. 


5. CONCLUSION 


This study aims to contribute to the advancement of automated safety monitoring technology in the field of 
construction sites. The proposed method utilizes a Deep Learning-based OPENPOSE pose estimation model to 
estimate the postures of scaffold workers on construction sites. This estimation aims to provide real-time risk alerts 
to prevent falls from height (FFH) accidents. This research is expected to reduce the fatigue of construction site 
managers and decrease the occurrence rate of scaffold-related fall accidents. 


However, during this research, limitations and areas for improvement have been identified. Due to the nature of 
construction sites, there are often obstacles and situations where workers are obstructed or difficult to detect due 
to equipment. Additionally, as objects move farther from the camera, issues were observed in joint angle 
calculations and false positives/negatives of worker detection. This indicates the need for further research and 
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algorithm improvements. To address these challenges, future research aims to enhance detection accuracy based 
on distance and devise strategies to detect workers obstructed by obstacles. Furthermore, incorporating Multi- 
Camera setups is intended to calculate worker head angles. 
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